Skip to content

Add automotive reference implementation workflow #407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 96 commits into from
May 26, 2025

Conversation

anandhu-eng
Copy link
Contributor

No description provided.

@anandhu-eng anandhu-eng requested a review from a team as a code owner May 13, 2025 15:45
Copy link
Contributor

github-actions bot commented May 13, 2025

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@anandhu-eng anandhu-eng marked this pull request as draft May 13, 2025 15:58
@anandhu-eng anandhu-eng marked this pull request as ready for review May 13, 2025 19:58
@anandhu-eng
Copy link
Contributor Author

As synced, lets convert this PR to draft and once ready with test actions, we can merge this PR.

@anandhu-eng anandhu-eng marked this pull request as draft May 13, 2025 20:12
@arjunsuresh
Copy link
Collaborator

@anandhu-eng good to merge?

@anandhu-eng anandhu-eng marked this pull request as ready for review May 25, 2025 04:38
@anandhu-eng
Copy link
Contributor Author

anandhu-eng commented May 25, 2025

Hi @arjunsuresh , The Bevformer CPU workflow is running fine. I m getting an accuracy of 0.268 mAP which is comparable to the result taken by Rod - 0.2683556 mAP. There is some issue in summary table - results not getting populated in latency and accuracy columns. We can sync and raise that as a separate PR.

Edit: The following run cmd was used in testing:

mlc run script --tags=run-abtf-inference,reference,_v0.5,_full --model=bevformer --docker --docker_cache=no --quiet --env.MLC_USE_DATASET_FROM_HOST=yes --env.MLC_USE_MODEL_FROM_HOST=yes --device=cpu --implementation=reference --framework=onnxruntime --scenario=SingleStream --batch_size=1

@anandhu-eng anandhu-eng changed the title Add automotive 0.5 model download scripts Add automotive reference implementation workflow May 25, 2025
@anandhu-eng anandhu-eng marked this pull request as draft May 26, 2025 05:50
@anandhu-eng anandhu-eng marked this pull request as ready for review May 26, 2025 07:51
@arjunsuresh arjunsuresh merged commit 9553d0a into mlcommons:dev May 26, 2025
76 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants